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of the same variables used to compute complexity were also used to predict PALT, those 
variables were verified as possible predictors of cost by building a prediction model for 
complexity score. The following variables served as good predictors of PALT: contract 
amount, contract description and contract type. It was also determined that the 
competitive nature of the contract had little impact on PALT. With this data, it is 
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I. INTRODUCTION 


A. BACKGROUND 

The Procurement Department of the Naval Air Warfare Center Weapons Division 
(NAWCWPNS) at China Lake, California awards contracts of all types in support of the 
current needs of the U. S. Navy. Business is conducted through a decentralized 
department with seven buying divisions in various locations around the center. Through 
their cognizant procurement office, technical department personnel of NAWCWPNS, 
may request contracts to fulfill their missions. These same technical departments 
currently pay into a combined overhead account to fund the procurement department. 
This funding accounts for approximately ninety percent of the budget of the procurement 
department. The remaining funding is generated through the charging of the cost of 
direct hours. The cost of labor is the major cost incurred by the procurement 
department. (Telephone call, NAWC, 31 August 1992) 

Most recently, however, the Department of Defense has enacted a set of rules and 
guidelines referred to as Defense Business Operations Fund (DBOF). Although DBOF 
was implemented 1 October 1991, it is not apparent what the full implications of this new 
guidance will be. As implementation of DBOF proceeds, it appears that the resulting 
financial system will require that Department of Defense activities, such as 


NAWCWPNS, China Lake, conduct business in an increasingly cost effective manner. 


General guidance concerning DBOF is contained in the following excerpt from the 


Defense Management Report Decision, number 971. 


DEFENSE BUSINESS OPERATIONS FUND: To improve the tools available to 
the managers of the support establishment, the financial system should collect all 
of the costs related to an output. In addition, the requirements on the level of 
performance and support required of the support establishment should be 
established by its customers, and reflected in the prices charged those customers. 
Expanding the use of cost accounting principles, and performance and activity 
based budgeting in a newly established "DEFENSE BUSINESS OPERATIONS 
FUND" (DBOF) should provide the basic building blocks to achieving the goals 
described above. 


However, before a business area can be included in the DBOF it must first meet 
three requirements: 1) identify the output of that business; 2) identify customers of that 
business; and 3) develop a cost accounting system that relates cost to those outputs. 
(DMRD, nr.971, JAN 1992) 

In response to these requirements, the identity of output for the procurement 
department at NAWCWPNS, China Lake is a contract and their customers are the 
technical departments. To develop a workload measurement and staffing tool for the 
procurement department, a Workload Management Impact Team (WIT) has been formed. 
This team has developed standards for determining the complexity of prospective 
contracts. These accounting changes required by DBOF may reveal which agency 
Operates most cost effectively. However, until DBOF is better understood by the 


Department of Defense agencies, the Naval Air Weapon Center, China Lake continues 


to operate under the existing Department of Defense and Navy Policy. (NAWCWPNS, 
February 1992, pp.1-6) 

As mentioned earlier, the technical departments located at NAWCWPNS, China 
Lake currently pay a percentage of their budgets toward an overhead fund for the support 
departments, including the procurement department. Along with the changes required 
in DBOF, these same technical departments will begin paying only the procurement costs 
associated with the contracts requested by their departments. It is even anticipated that 
in the future, DBOF may allow for contracts to be procured through other than China 
Lake offices in an effort to obtain procurement services at the lowest cost. The desire 
of the procurement department is to become more competitive in the process of awarding 
a contract so their potential customers will view them as a cost effective alternative. 

There are reasons why the procurement department at NAWCWPNS, China Lake 
is already an attractive alternative compared to other procurement sources. One reason 
is the proximity of on-site procurement offices to the laboratories and ranges where the 
technical departments are located. This closeness allows an increase in the monitoring 
of contracts and convenience for writing new contracts. Another reason is the high level 


of customer service to which the technical departments have become accustomed. 


B. PROBLEM DESCRIPTION 
The Department Head for Procurement at Naval Air Warfare Center 


(NAWCWPNS), China Lake, California has determined that in order to improve their 


competitiveness, a model which provides an accurate prediction of the dollar cost amount 
and the time spent in awarding a contract must be developed. 

So far, a computer based model for the calculation of predicted contract cost is 
being developed by the procurement department. This model uses as its input the same 
variables that the regression models developed in this thesis use, yet this model treats 
these variables differently. It assigns point values obtained from the WIT to the levels 
of these variables. Each contract then receives points that reflect the complexity involved 
with writing that contract. For instance a proposed contract with an estimated amount 
of one hundred thousand dollars receives one hundred complexity points for contract 
amount, whereas a contract written for ten million receives three hundred points for 
complexity. This model then takes these terms and sums the point values. This total is 
used as a multiplier of a basic number of hours, in order to forecast the administrative 
cost of awarding that contract. (NAWCWPNS, 20 April 1992) 

These predictions would be extremely helpful for budgetary planning, both for the 
procurement department and for those departments or “customers”, requesting the 
contract. This information would be provided to the potential customer by procurement 
personnel during the initial discussion of the requirement. The primary goal is to 
determine how much a single contract costs and the secondary goal is to determine the 
number of days it takes to award a contract. 

This competitive edge, though seemingly slight, can manifest itself in hundreds of 
thousands of dollars annually. The Procurement Department annually processes in excess 


of 42,000 simplified purchase actions ($25,000 and under) for a value in excess of 


$60,000,000. In addition, more than 6,000 large purchase actions are processed yearly, 
resulting in contracts, delivery orders, or modifications to existing contracts currently in 
excess of $258,000,000 annually. (NAWCWPNS, April 1989, p.1-2) 

Thus far, little data based modelling, has been conducted to forecast these values. 
A model is required that can use the existing, explanatory data to accurately predict the 
cost of awarding a contract both in terms of time and of money. Unfortunately, no cost 
data is available, so little can be done to predict the dollar cost of awarding a contract. 
The reason for this deficit in data is the cost and the man-hours required to collect such 
data. The costs of procurement are also difficult to quantify. It is the task of the 
Workload Management Impact Team (WIT) to determine what work can be measured 
and the relative complexity, (and thus cost), of that work. These experts’ knowledge is 
captured in a complexity score that is based on cost. (NAWCWPNS, 17 April 1991) 

The complexity score will be used by procurement personnel to determine cost by: 
1) determining actual average. cost of a baseline contract, and 2) multiplying this cost by 
the complexity score to get a cost estimate for the contract of interest. 

Although little data is available regarding procurement cost, much data is available 
concerning the time it takes to award a contract. This data is located in chronological 
sequence, 1989 through 1991, in Appendices A, B, and C. The number of days between 
the approval of the acquisition requirements package (ARP) by the respective 
procurement division and the final award of a contract is referred to as Procurement 
Administrative Lead Time (PALT). Although PALT includes the high expense direct 


labor hours, it also includes that time when a contract remains unprocessed and static. 


Therefore, PALT in itself is an imperfect measure of the time required to process a 
contract. In the near term, a multivariate regression model is desired that can predict 
PALT, so that the time spent in writing a contract can be forecast. 

When cost data becomes available, a similar model can be developed for that 
factor. In the interim, the assumed positive correlation with PALT can be exploited to 
predict that the relative cost of contracts will be in the same order as their predicted 
PALT values. 

The process undergone by each contract is lengthy and complex. A very 
abbreviated example of the process would, however, be of some informational use at this 
point. The process begins when a technical department requests a specific supply or 
service. The procurement department formalizes a solicitation, which includes a 
statement of work or a specification along with prospective contractual terms and 
conditions, which is then made available to perspective contractors. The contractors then 
submit their proposals, the government evaluates the offers, and after the negotiation a 
contractor is awarded the contract. 

A detailed representation of this process is found in the flowchart contained in 
Appendix D. The WIT has assigned points that reflect labor cost at each phase. These 
are accumulated into a “complexity” score for each contract. It is possible that this 
complexity score may be a better response variable for fitting a cost model than PALT, 


because it does not consider the dead-time that sometimes occurs. 


C. SCOPE 

The analysis in this study is limited to large purchases occurring at the Naval Air 
Weapon Center, China Lake, California within the past four years. Data on Procurement 
Administrative Lead Time (PALT) was selected beginning in 1989 because of 
accessibility. Also, considering that regulatory changes are often implemented which 
affect processing methods that tend to increase PALT, it was desirable to use recent 
fiscal year data. 

Though available, data from 1992 was not included because it might not accurately 
represent the entire year. The reason for this misrepresentation is that the technical 
departments tend to submit their requirements at the midpoint of the fiscal year or later, 
resulting from a late receipt of budgetary information. This causes an increase in the 
number of contracts awarded in the last quarter of the fiscal year, which in turn may 
cause a seasonal rise in PALT. The increase in the number of acquisition requirements 
packages being processed leads to an increased workload for the procurement department 
and therefore increases the bottlenecks in the procurement process. As a result, a 
contract processed in the beginning of the fiscal year may have a decreased PALT 
compared to the same contract processed in the end of the fiscal year. Therefore, the 
entire years of 1989, 1990 and 1991 were considered for this study. 

Additionally, the years of 1989 and 1990 are grouped together to develop the 
quadratic regression model required for predicting PALT. This model is then used to 


predict 1991 PALT to verify the model. 


This chapter has defined the problem, its motivation and its background. The 
scope of the research and the intended solution have also been described so that the next 


step, exploratory data analysis, can be taken. 








HW. METHODOLOGY 


A. INITIAL ANALYSIS 

As often happens, some of the data available for analysis is continuous and some 
is categorical, as shown in Appendices A through C. Data for each contract included the 
following continuous variables: contract amount, complexity score and PALT. Contract 
amount, given in dollars, is the total amount expected to be paid for the particular supply 
or service. Complexity score, given in points, refers to the difficulty in processing that 
particular contract. This point system was initially developed by the procurement 
department to investigate the effort required to accomplish a division’s workload so that 
staffing levels could be determined. Additionally, this point system is being used to 
determine the relative cost of writing a contract. 

These points are based in part on the same variables used in this study. 
Procurement Administrative Lead Time (PALT), given in days, was also treated as 
continuous. The exploratory data analysis indicated that PALT has large variability 
(spread > one year) at almost every level of each predictor when viewed bivariately with 
PALT. It also revealed that the contract amount values are dense at the low end and 
very spread out at the large end. A common procedure that is used to examine this type 
of data is to take the natural logarithm to spread the data more evenly to facilitate 
analysis. Once this was done the model fitting proceeded. Figure 1 shows this spread 


along with a fitted LOWESS curve indicating the average rise in PALT for increasing 


contract amount. The term LOWESS stands for locally weighted regression scatter plot 
smoothing, (Chambers, 1983, p.94). In other words it shows an average value at any 


point along the curve. 


PALT VS LOG AMOUNT WITH LOWESS CURVE 
1989-1990 DATA 


PALT 





LOG AMOUNT 


Figure 1. Scatter plot: PALT vs. Log Amount with LOWESS curve 1989-1990 


Figure 2 uses a number of box plots to explain the range of PALT for different 
levels of contract amount. Each box plot is made up of a box with a center line and 
small circle generally located within the box. The top and bottom edges of the box 


indicate upper and lower quartiles of the data. The median is indicated by the interior 
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line passing within the box. The circle within the box denotes the mean value of the 
data. The lines that extend from the box indicate the tails of the data that lie outside the 
inner quartile range and their length is equal to 1.5 times the inner quartile range. 
Values outside these outer lines are denoted first by hollow circles then by filled circles. 
(Chambers, 1983, p.21) 

Figure 2 reveals that PALT varied greatly over different contract amounts, 
generally for more than a year’s time. This graph also reveals that as amount increases 


so do the mean and vaniance of PALT. 


STRIP BOX PLOT 
PALT VS LOG AMOUNT 
1989-1990 


600 


400 
ee 


PALT 


200 


ii 12 13 14 
LOG AMOUNT 


Figure 2. Strip box plot: PALT vs. Log Amount 1989-1990 
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Categorical variables included in this study included contract type, contract amount, 
contract description, and competitive nature. Contract description is separated into three 
self-explanatory categories: service, supply, and research and development. Figure 3 
shows the average PALT associated with the three levels of contract description. The 


number of contracts for each type is: research and development - 70, supply contracts - 


450, and service - 62. 


STRIP BOX PLOT 
PALT VS CONTRACT DESCRIPTION 


600 


PALT 
400 


200 





R AND D SUPPLY SERVICE | 


Figure 3. Strip box plot: PALT vs. Contract Description (1989-1990) 
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Contract type is separated into two categories, cost reimbursable and fixed price. 
To explain, a cost reimbursable contract has as its contract amount a ceiling figure, up 
to which the contractor may spend. For example, if a variation arises in the number of 
tests required for the contract, no modification to the contract would be required as long 
as the contract amount is not exceeded. Conversely, the fixed price contract features a 
firm price which will be paid to the contractor upon acceptance of the supply or service. 
Figure 4 represents the 1989-1990 PALT values associated with the two levels of 
contract type. The number of contracts for each level is: cost reimbursable - 69, and 


fixed price - 490. 


STRIP BOX PLOT 
PALT VS CONTRACT TYPE 1989-1990 
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PALT 
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° 
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Figure 4. Strip box plot: PALT vs. Contract Type 1989-1990 
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Finally, the competitive nature of each contract was noted. A contract is either 
awarded on a competitive basis, or it is awarded on a sole source basis, that is, there is 
only one source that can satisfy the government requirement. The number of contracts 
for each competitive nature 1s: competitive - 413, and sole source - 76. (NAWCWPNS, 
6 November 1989) 

The data was only available in typed form. It was scanned and edited into 
workable form using a word processor (Word Perfect 5.1, User’s Manual, 1991). Once 
complete, the data was next sent as an ASCII file to a data manipulation program to 
enable the use of other mathematically based software packages (CSS, User’s Manual, 


1990). It was in this form that final editing took place and data analysis began. 


B. ASSUMPTIONS 

After the data was initially reviewed, three outliers were detected in the 1991 data 
in the PALT field. The average PALT value for that year is 147.4 with a standard 
deviation of 98.554. These three PALT values were 8018, 8032 and 8146 days, 
approximately twenty-two years each, for contract numbers: N6053090R002, 
N6053090C0118 and N6053090C0298, respectively. After consultation with the 
procurement department at NAWCWPNS China Lake, the three values were dropped 
under the assumption that they were administrative errors. (Telephone conversation, July 
1) 

Also, contracts whose type was time and material / labor hour (TM/LH) were not 


considered because of the small number of contracts they represent, (Telephone 
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conversation, NAWCWPNS, August 1992). This type of contract comprised less than 
One percent of the contracts analyzed. 
Therefore, total number of contracts analyzed from the grouped years of 1989 and 


1990 was 559 , and the number from 1991 was 275. 


C. MODEL TYPE 

The model used to predict Procurement Administrative Lead Time (PALT) must 
be able to utilize both continuous and categorical variables as described earlier. 
Exploratory data analysis revealed that a quadratic function of contract amount should 
be included in the model. (This subsequently resulted in better full model fits for 1989- 
1990 and 1991 data, and did a better job of predicting 1991 PALT using a fitted model.) 
Therefore, a multivariate, regression model was fit with a quadratic continuous variable 
and indicator variables for the levels of the categorical variables. It was implemented 
in the MINITAB software package as an analysis of covariance so that interactions 
between the continuous and categorical variables could be revealed. The following 
example shows how an analysis of covariance model is formed. It is a full model with 
one categorical variable at m levels and one continuous variable. (Fountain and Ward, 


1992, p.8) 
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Yi = By +BiX, + 6%, + RR ooo 2 eee eee 
where 
1 ; if observation i is from level j of the treatment 


QO ; otherwise 


W.: = x 


ij ji  * Zi ; 

where Y represents the response being measured (PALT), 
6 represents the regression coefficient for that categorical variable X, 
Z is a continuous variable (contract amount or complexity score), 
w is an interaction effect that uses two or more variables, 


11s the observation number, 


_j is a level of categorical variable X. 


The exploratory data analysis helped ensure that an appropriate model would be 
developed. Through the use of graphical methods, PALT was shown to be highly 
varied. This meant that PALT would be difficult to forecast. Also, no single variable 
could be identified as having relatively strong predictive capability. Therefore, all 


variables would have to be considered for use in the model. 
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I. MODEL DEVELOPMENT 


A. MODEL TO PREDICT PALT 

A parsimonious model that accurately predicts PALT using only a small set of 
explanatory variables is desired. Only those variables that are useful for prediction 
should be included since a small model is easier to understand, and data for a small 
model is easier and less expensive to collect. Also, a large model, with more 
explanatory variables, produces a smaller Sum of Squared Errors (SSE), but the 
additional variables may result in an increased Mean Square Error (MSE), meaning that 
predictions made with the model are less precise (greater variance). MSE=SSE/(n-p-1), 
where n is the number of observations, and p is - number of explanatory variables in 
the model. As additional variables are added, the numerator (SSE) decreases, but so 
does the denominator. If the decrease in SSE is not enough to offset the decrease in 
(n-p-1), MSE will increase. To obtain a parsimonious model, the full model utilizing all 
the variables and all possible interactions, was first fit. Then, terms that were deemed 
to be poor predictors were sequentially eliminated. As each model was fit, estimates of 
the coefficients (G’s from the model expression above) were calculated for the terms 
included in the model. These estimates are random variables. Assuming that the PALT 
values are realizations of normally distributed random variables with constant variance, 
the estimates are also normally distributed for each coefficient, this distributional 


assumption was use to calculate the probability that an estimate as far from zero as the 
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one realized would obtain if the true coefficient were in fact zero (that is, if the 
corresponding explanatory variable had no predictive power). These probabilities were 
reported as p-values. At each iteration the term with the largest p-value was eliminated 
until no term with a p-value greater than 0.10 remained. 

As stated, the p-values are based on the assumption that PALT is a normally 
random variable. Exploratory data analysis indicates that this is not the case. Normally 
distributed, random variables are symmetric, but the exploratory data analysis revealed 
that the observed PALT has a right skew. This is not surprising since PALT is a non- 
negative variable virtually unbounded. A power transformation, SYMPALT= PALT™%, 
was utilized to reduce this right skew. The symmetry and normal probability plots 
showed a reduction in skew, but the full model utilizing this term does not fit as well for 
the 1989-1990 data, and does not predict either data set as well. Since the goal of this 
study is to predict PALT, this transformation of the data was not used. Thus the p- 
values should not be used for formal hypothesis tests about the regression coefficients, 
but they can be used to identify variables that are candidates for elimination. 

Another transformation did however assist in the fitting of the model. The log 
of each contract amount was squared and used in the model. Numerically, the 
correlation between actual and predicted PALT for the model without the squared log 
amount is 0.492, while the same correlation for the model containing the squared term 
is 0.523. Therefore, the squared log contract amount was used. The terms listed in 
Table 1 below, are the best choices in terms of a well-fitted model. The expression 


"level" refers to the different possibilities each categorical variable might contain. Actual 
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levels used are located inside the box next to each categorical variable. The table lists 


all values of the fitted regression model. 


TABLE 1. FITTED REGRESSION MODEL USING 1989 - 1990 DATA 


— 


[Constant | | 53.7650 | 27.03 | «1.99 | 0.047 
[Contract Type | cR___ | -sa.asan_[27.03 | 2.16 | 0.01 


Contract R&D -46.4607 8.277 -5.61 0.000 
Description 
Contract SUPP 19.8525 8.133 -2.44 0.015 
Description 
(In (Contract 1.4648 0.1574 
Amount))’ 
C 


Contract type* R -29.1581 8.277 -3.52 fen 000 
Contract R&D 
Description 
Contract type* CR 23.0586 8.133 0.005 
Contract SUPP 
Description 

CR | 


(In (Contract 
Amount))” * 
Contract Type 





Competitive nature, a categorical variable, is not used because it had little 
predictive power at either of its levels. 

The term Coeff refers to the coefficient multiplied by the variable it represents. 
However if the variable type is not present the Secfficient is multiplied by zero. The 


only term that is always present in the calculation of PALT is the constant term. 
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The reason some levels of the categorical variables appear, and others do not, is 
because that absent level of each of these variables has been assumed as part of the 
constant term. ‘Therefore, the categorical variables shown intrinsically assume the 
existence of one more level. For instance, contract type shows cost reimbursable but not 
fixed price, because fixed price contracts were chosen to be assumed into the constant 
term. Any contract with contract type at the cost reimbursable level results in 
-58.4521 days toward the prediction of PALT. If two contracts are identical except for 
the contract type, the cost reimbursable contract will have a predicted PALT that is 
58.4521 days shorter. 

The following is an example of how PALT is predicted through the use of this 
model. If a fixed wate sole source, supply contract were written for the amount of 


$1,000,000 its prediction of PALT would equal: 


-53.7650+0*(-58.4521) +0*(46.4607) +1*(19.8525) + (In(1 ,000,000))2*(1.4648) + 
0*(-29.1581) +0*1*(23.0586) + (In(1,000,000))2*0*(0.4286) 


= 245.67 days of PALT 


B. COMPLEXITY MODEL 
To take a preliminary look at the predictability of cost using the available 
predictors, without cost data, a model was built for complexity score using the 1989-1990 


data to predict 1991 complexity scores. The complexity point totals for the 1989-1990 
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data were put through this fitted regression model. This model also utilized the squared 


log amount for one of its explanatory variables. Results are given in Chapter IV. 


c. ALTERNATIVE MODEL 

Another fitted model was developed in order to determine if better predictive 
capabilities could be produced. This model used only those contracts whose PALT was 
less than or equal to one year. The predictive capability of this model was not as strong 


as the model using all of the data, therefore, the model was not analyzed any further. 


Thus far, the full model was reduced and fitted to produce the best predictions 
of PALT. An example helped to show how the model is used. The complexity model 
was also fitted to reveal predictability of complexity points. The analysis of the results 


from these models follows. 
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IV. ANALYSIS OF RESULTS 


A. PREDICTED VS. ACTUAL PALT FOR 1989-1990 

An upper bound on the accuracy that could be expected when using the fitted 
model as a predictor was determined by forecasting PALT for the 1989-1990 data, 
having derived the model from this same data. Because 1989-1990 data was used to 
develop the fitted model, no ensuing prediction of PALT from the following year should 
be more accurate than the predicted PALT for the year upon which the model was based. 
The results of comparing predicted to actual values of PALT showed that it is a difficult 
variable to predict. The Pearson product moment correlation coefficient (r-value), from 
this analysis is 0.523. Therefore, this model explains just over one-quarter, (R’ = 
0.274), of the variability found in PALT. The term, r-value, refers to the amount of 
linear correlation between the two variables considered. The next section gives the 


results of using this model to predict 1991 PALT. 


B. 1991 PALT PREDICTED USING 1989-1990 FITTED MODEL 
The following table will help to illustrate the effect of each of the variables listed 
as having a significant predictive capability toward PALT. The p-value, as explained 


earlier, was shown to be below 0.1 for all variable combinations. 


ipa 


TABLE 2. STATISTICAL ANALYSIS OF THE FITTED MODEL 


| Contract Description 172242 108070 Ser 0.000 
(Log Contract Amount)? 653516 593169 86.56 0.000 


Contract Type* 2 94628 50622 0.001 


Contract Description 


Contract Type* yt 50353 50353 GBS 0.001 
(Log Contract Amount)’ 
55 - 


Total | 554 | 4973204 | - 









q 





The results from this analysis reveal the predictive capabilities of the fitted model. 
The r-value derived in comparing actual 1991 PALT to predicted PALT, having used 
the fitted model, is 0.356. This low value confirms the exploratory data analysis that 


PALT is a difficult variable to forecast. 


OF COMPLEXITY MODEL 

In the long term, NAWCWPNS, China Lake wants to predict the administrative 
cost of writing a contract. But, no cost data are available. It is believed, however, that 
cost should be more predictable than PALT since only the time actually spent working 
on the contract is included in the cost. A preliminary examination of this issue was 
conducted by fitting a second multivariate regression model to the available data using 


the complexity score as the response variable. Using the 1989-1990 data, this model was 
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used to predict 1991 complexity scores. The results were very good. The correlation 
coefficient for predicted score and actual score was 0.825. If the assumed relationship 
between complexity score and cost is true, good predictions should result from a model 
based on currently available data items. Although the data available was useful more 
continuous data would help to predict complexity score. The accuracy obtained in 
forecasting PALT was low, however, the complexity model used the same explanatory 
data with better results. Both PALT and complexity score could be better forecast, 


however, if more data were available. 
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V. CONCLUSIONS 


A. PALT 

There exists a significant relationship between Procurement Administrative Lead 
Time (PALT) and: contract type, contract amount and contract description. However, 
the competitive nature of the contract had little impact on predicting PALT. PALT is 


a difficult variable to forecast with precision. 


B. COST 

Once cost data is recorded, actual cost should be statistically predictable for the 
purposes of the Defense Business Operations Fund (DBOF). This conclusion is based 
on the predictability of the complexity score, and because while the PALT data exhibited 
a high standard deviation, the model lost very little explanation of variability when the 


prediction for 1991 was made using a fitted model based on 1989-1990 data. 


OF ADDITIONAL DATA 
Once cost data is accumulated, the model should get increasingly accurate in its 
prediction of PALT as well as cost. The collection of cost data may, unfortunately, 


cause PALT to increase. 
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VI. RECOMMENDATIONS 


A. DATA COLLECTION 

The first recommendation is that award date should be included as part of the 
regular data base. This will further categorize the contracts into monthly segments. In 
doing this, forecasting PALT and eventually the cost of awarding a contract, will be 
facilitated by making available one more explanatory variable for the analysis. This will 
enable a thorough analysis of seasonal differences for both PALT and cost. 

The second recommendation is to continue recording the complexity scores along 
with all other data entries. This will ensure the constant tracking of this variable to 
ensure accuracy in assigning points to bats attributes. 

The third recommendation is that cost data be taken over a wide range of different 
contracts according to the following minimum data collection plan. This plan will ensure 
that all types of contracts are considered without missing a specific type of contract. An 


entry of one indicates the category of contract to be recorded. 
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TABLE 3. DATA COLLECTION PLAN FOR COVERAGE OF ALL CONTRACT 
TYPES 





If this plan is followed, the recorded data should represent a complete coverage 
over the range of contracts. This plan may be expanded, for example, to ensure proper 
coverage of all categories of contract amounts simply by including all of the above 
contracts for each of the amount categories. Since twelve different combinations exist 
in the above plan and currently contract amounts are separated into five groups, sixty 


different contracts would be necessary to cover all the possible combinations. By 


a 


collecting data over the different categories of contract amount, the variance of the 
updated model should be reduced. 

Since collecting data is costly and time consuming, data should be collected on 
several baseline contracts and especially on those contracts whose levels appeared 
important for prediction. 

The correlation coefficients between predicted and actual PALT, revealed that 
the fitted, PALT model was able to explain over one quarter of the variance of the data. 
This statistic could be improved by including more variables in the data base. These 
variables include, but are not limited to those used to accumulate points for complexity. 
The recommendation is therefore made that the same variables used to determine 
complexity scores, be made available to determine PALT. Point assignments are 


available in Appendix E. 


B. MODEL UPDATES 

The fitted, PALT model should be updated upon the completion of the cost data 
collection over at least one year. At that point a cost model should also be developed. 
Once this is done both models should be updated regularly (e.g. every two years) to 
verify their accuracy. The model presented in this study grouped two years together to 
build a model in order to predict a third. This method was necessary and seems 
reasonable in view of current economic trends. If too many years are used as a group, 
the opportunity remains for the model to become out of date with current time and cost 


considerations. 
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Using these recommendations on data collection and data use will help determine 


the most accurate predictions of cost and number of days required to write a contract. 
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Contract Number CT 


N6053089C0002 
N6053089C0003 
N6053089C0005 
N6053089C0006 
N6053089C0008 
N6053089C0009 
N6053089C0010 
N6053089C0014 
N6053089C0015 
N6053089C0016 
N6053089C0017 
N6053089C0018 
N6053089C0019 
N6053089C0020 
N6053089C0021 
N6053089C0025 
N6053089C0026 
N6053089C0029 
N6053089C0032 
N6053089C0033 
N6053089C0036 
N6053089C0037 
N6053089C0038 
N6053089C0040 
N6053089C0041 
N6053089C0042 
N6053089C0043 
N6053089C0044 
N6053089C0045 
N6053089C0046 
N6053089C0048 
N6053089C0051 
N6053089C0052 
N6053089C0053 
N6053089C0056 
N6053089C0060 


FP 
FP 
FP 
FP 
FP 
lees 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
Je 
FP 
FP 
CR 
| 
BE 
a 2 
FP 
FP 


FP 
FP 
FP 
FP 
FP 


APPENDIX A: 1989 DATA 


Desc. 
SERV 
SUPP 
SUPP 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
RD 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SERV 
SERV 
SERV 
SUPP 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


Amount 
259284.50 


$2343 


38021895 


120144 
52725 


30270.98 
8340000 


121374 
100376 
73315 
45500 
63960 
30750 
67900 
49785 
413900 
38622 
248217 
49926 
37155 
49990 
54082 
48854 
105500 
571619 
42350 
49979 
50000 
24648 
46417 
57255 
29500 
98618 


45281.50 


46777 
29570 


30 


CN 
COMP 
COMP 
COMP 
COMP 
COMP 
SS 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
SS 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 


PALT 
22 
189 
78 
Loe 
120 
121 
349 
165 
147 
190 
85 
142 
137 
144 
206 
357 
205 

147 


Score 
100 
115 
100 
115 
100 
135 
315 
100 
100 
100 
100 
100 
100 
100 
100 
135 
100 
135 
115 
100 
200 
70 
200 
100 
150 
100 
100 
100 
100 


N6053089C0061 
N6053089C0062 
N6053089C0063 
N6053089C0064 
N6053089C0065 
N6053089C0066 
N6053089C0067 
N6053089C0070 
N6053089C0071 
N6053089C0075 
N6053089C0081 
N6053089C0083 
N6053089C0084 
N6053089C0085 
N6053089C0086 
N6053089C0088 
N6053089C0090 
N6053089C0092 
N6053089C0093 
N6053089C0094 
N6053089C0095 
N6053089C0096 
N6053089C0097 
N6053089C0098 
N6053089C0099 
N6053089C0100 
N6053089C0101 
N6053089C0102 
N6053089C0103 
N6053089C0107 
N6053089C0111 
N6053089C0113 
N6053089C0114 
N6053089C0115 
N6053089C0117 
N6053089C0121 
N6053089C0125 
N6053089C0128 
N6053089C0129 
N6053089C0132 
N6053089C0133 
N6053089C0134 
N6053089C0137 


CR 


CR 


FP 
FP 
CR 
FP 
FP 
CR 
FP 


FP 
IP 
FP 


FP 
FP 
FP 
FP 


CR 
FP 
FP 
FP 
F 


ae, 


TRU UUyUU 


FP 


SUPP 
SERV 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SERV 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
RD 

SUPP 
SUPP 
SUPP 
SUPP 
RD 

SUPP 
SUPP 
SUPP 
SERV 
SUPP 
SUPP 
SUPP 
SERV 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


NEES, 
243268 
200684 


43087.25 
311107.74 


162248 


2442398 .47 


49800 


86585.36 


395585 
62543 
113691 
40300 
53478 
29501/ 
940000 
39ng9 
40500 
77380 


76278.72 
291692 


81926 
30025 
49255 
428275 
457126 
40000 
76392 


1164672 
55366.80 


40736 
395000 
67120 
87321 
141420 
Io 7531 
232778 
86940 
94490 


31761.60 
4248817 
1500000 


49750 


31 


N6053089C0139 
N6053089C0140 
N6053089C0144 
N6053089C0145 
N6053089C0147 
N6053089C0148 
N6053089C0151 
N6053089C0155 
N6053089C0156 
N6053089C0158 
N6053089C0162 
N6053089C0164 
N6053089C0165 
N6053089C0166 
N6053089C0171 
N6053089C0176 
N6053089C0177 
N6053089C0178 
N6053089C0181 
N6053089C0182 
N6053089C0183 
N6053089C0185 
N6053089C0186 
N6053089C0188 
N6053089C0189 
N6053089C0190 
N6053089C0194 
N6053089C0196 
N6053089C0197 
N6053089C0198 
N6053089C0199 
N6053089C0201 
N6053089C0202 
N6053089C0204 
N6053089C0205 
N6053089C0206 
N6053089C0208 
N6053089C0209 
N6053089C0210 
N6053089C0211 
N6053089C0212 
N6053089C0214 
N6053089C0215 


FP 


FP 


SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


SUPP 
SERV 


SUPP 
SUPP 


SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


SUPP 
SERV 


SUPP 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


87674.40 


50203 
49600 


85465.80 


150015 
124556 


39605.70 


63854 


1200000 


35074 


1220708 


273070 


74210.12 


450000 
500000 


36312.25 
65308.57 


67500 


146808.46 
53628.86 
1941308 


85960 
74173 
246392 
75615 
109040 
137384 


JI 9S 


294000 
36000 
98575 
71824 
49255 
65000 
42250 
57140 
38564 
32500 


1355162 


30690 
110500 
63184 
49655 
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A 


— 


N6053089C02 16 
N6053089C0217 
N6053089C0219 
N6053089C0220 
N6053089C0221 
N6053089C0222 
N6053089C0223 
N6053089C0224 
N6053089C0225 
N6053089C0229 
N6053089C0233 
N6053089C0234 
N6053089C0235 
N6053089C0236 
N6053089C0240 
N6053089C0241 
N6053089C0247 
N6053089C0250 
N6053089C025 1 
N6053089C0252 
N6053089C0253 
N6053089C0254 
N6053089C0255 
N6053089C0256 
N6053089C0258 
N6053089C0259 
N6053089C0261 
N6053089C0263 
N6053089C0265 
N6053089C0268 
N6053089C0270 
N6053089C0271 
N6053089C0272 
N6053089C0276 
N6053089C0278 
N6053089C0282 
N6053089C0284 
N6053089C0288 
N6053089C0291 
N6053089C0295 
N6053089C0297 
N6053089C0298 
N6053089C0299 


FP 


CR 
FP 


soo 


FP 


IRUBUV UU 


SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


247966 
549013 
49993 
54014 
249911 
89970 
35100 
108276 
30990 
77844 
76233.71 
50951 
120762.70 
37000 
32 176.80 
49556.60 
28850 
119000 
32400 
Zao”) 
1131952 
29544 
Bibi)! 
749000 
32650 
78800 
780708 
46297 
165506.25 
35000 
116390 
29302 
876892 
33955 
47250 
34173 
330358 
177027 
452500 
139800 
29525 
666041 
34717 


eX) 


N6053089C0302 
N6053089C0303 
N6053089C0304 
N6053089C0306 
N6053089C0307 
N6053089C0308 
N6053089C03 10 
N6053089C0311 
N6053089C03 12 
N6053089C0313 
N6053089C0314 
N6053089C0315 
N6053089C0317 
N6053089C0318 
N6053089C0319 
N6053089C0320 
N6053089C0321 
N6053089C0324 
N6053089C0325 
N6053089C0326 
N6053089C0327 
N6053089C0332 
N6053089C0333 
N6053089C0336 
N6053089C0337 
N6053089C0338 
N6053089C0341 
N6053089C0344 
N6053089C0345 
N6053089C0346 
N6053089C0347 
N6053089C0349 
N6053089C0351 
N6053089C0353 
N6053089C0354 
N6053089C0355 
N6053089C0356 
N6053089C0358 
N6053089C0359 
N6053089C0360 
N6053089C0363 
N6053089C0364 
N6053089C0367 


FP 
FP 


FP 
FP 
FP 
FP 
CR 
CR 
CR 
FP 
FP 
FP 


FP 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
CR 
|e be 
FP 
CR 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
CR 
CR 
FP 
FP 
FP 
FP 
FP 
FP 


SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
RD 

SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SURE 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


SERV 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


153931 


108979.20 


491113 
199305 
541538 
135037 
111322 
166844 


43828.37 


490068 
66340 
40604 
90736 
43240 
35990 

316277 

155165 
36900 
44000 
75000 

998196 
89500 


30000 - 


50000 
207493 
31865 
29246 
174125 
440811 
26680 


5419492 
118720.80 


30240 
55541 
44200 


66472.91 


40510 
49840 
53735 
38750 
34000 
165296 
28950 


34 


N6053089C0370 
N6053089C0371 
N6053089C0373 
N6053089C0374 
N6053089C0375 
N6053089C0376 
N6053089C0378 
N6053089C0379 
N6053089C0380 
N6053089C038 1 
N6053089C0383 
N6053089C0384 
N6053089C0385 
N6053089C0388 
N6053089C0390 
N6053089C0391 
N6053089C0392 
N6053089C0393 
N6053089C0395 
N6053089C0397 
N6053089C0398 
N6053089C0399 
N6053089C0401 
N6053089C0403 


N6053089C0404-. 


N6053089C0417 
N6053089C0418 
N6053089C0420 
N6053089C0421 
N6053089C0423 
N6053089C0425 
N6053089C0426 
N6053089C0430 
N6053089C0431 
N6053089C0433 
N6053089C0439 
N6053089C0440 
N6053089C0441 
N6053089C0442 
N6053089C0444 
N6053089C0452 
N6053089C0456 
N6053089C0457 


FP 


SUPP 
SUPP 


SUPP 
SUPP 
SUPP 
SERV 
SUPP 
SUPP 
SUPP 


SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SERV 
SERV 
SUPP 
SUPP 


SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


73568 
34800 
200000 
60903 


80257.50 


35450 
382812 
32110 
175000 
43249 
90578 
70000 
71900 
51374 
67000 
34900 
195250 
74805 


69292.42 


58043 
61769 


25115 


76212 
60457 
69918 
80400 


26151.06 


57960 
41100 
45014 
45493 
184755 
166777 
31285 
336107 
42782 
79154 
165065 
47684 
30000 


92219.86 


97600 
47370 


35 


N6053089C0470 
N6053089C0471 
N6053089C0475 
N6053089C0489 
N6053089C0496 
N6053089C0497 
N6053089C0502 
N6053089C0511 
N6053089C05 12 
N6053089C05 16 
N6053089C05 17 
N6053089C05 19 
N6053089D0001 
N6053089D0013 
N6053089D0050 
N6053089D0059 
N6053089D0068 
N6053089D0082 
N6053089D0087 
N6053089D0089 
N6053089D0122 
N6053089D0126 
N6053089D0146 
N6053089D0195 
N6053089D0207 
N6053089D0226 
N6053089D0249 
N6053089D0290 
N6053089D0328 
N6053089D0352 
N6053089D0427 
N6053089D0432 


FP 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
CR 
FP 
FP 
CR 


Jellee 


CR 
CR 
FP 
CR 
FP 
CR 
FP 
FP 
FP 
FP 
CR 
| ee 
FP 
FP 
FP 
FP 
FP 


RD 
RD 
SUPP 
SUPP 
RD 
SUPP 
SUPP 
SUPP 
SERV 
SUPP 
RD 


69049 
62965 
46977 
34470 
29180 
65905 
81510 
19340 
45000 
91790 


1050860 
SUPP = 120000 
SERV 
SUPP 1280000 
SUPP 23000000 
SERV 115575 


SUPP 8371072 


66082.95 


SUPP 
SUPP 
SUPP 
SERV 
SUPP 
SUPP 
RD 

SUPP 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


593905 
940000 
67410 


1938217 
345151.62 
5987309 


83981 
34480 
418421 
104260 


2183160 


307043 
186000 
25000 


1160200 


36 


Contract Number CT 


N6053090C0002 
N6053090C0009 
N6053090C0012 
N6053090C0016 
N6053090C0017 
N6053090C0019 
N6053090C0021 
N6053090C0023 
N6053090C0029 
N6053090C0030 
N6053090C003 1 
N6053090C0032 
N6053090C0037 
N6053090C0039 
N6053090C0040 
N6053090C0042 
N6053090C0043 
N6053090C0045 
N6053090C0046 
N6053090C0047 
N6053090C0048 
N6053090C0049 
N6053090C005 1 
N6053090C0053 
N6053090C0056 
N6053090C0057 
N6053090C0059 
N6053090C0061 
N6053090C0062 
N6053090C 0063 
N6053090C0064 
N6053090C0065 
N6053090C0067 
N6053090C0072 
N6053090C0073 
N6053090C0077 
N6053090C0078 


FP 
CR 
FP 


one 


TIURGVVI UU 


FP 


APPENDIX B: 1990 DATA 


Desc. _Amount 
SUPP 34796 
RD 3415128 
SUPP _ 178700 
SUPP 111181.35 
SUPP = 163424 
SUPP 83554 
SUPP 193600 
SUPP 30009155 
SUPP 49200 
SUPP 49140 
SUPP 77350.80 
SERV 171684 
SUPP 572325 
SUPP 51050 
SUPP 27864 
SUPP 99600 
RD 46863 
SUPP 119000 
SUPP 52380 
SERV 383148.08 
SUPP = 135702 
SUPP 319824 
RD 492497 
SUPP 393360 
RD 49775.49 
RD 3672902 
SUPP 25594 
SUPP 67200 
SUPP 331948 
SUPP = 109200 
SUPP 77200 
SUPP 176620 
SERV 249968 
SUPP 54950 
SUPP 74955 
SUPP 241332 
SUPP 93749 


o7 


CN 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 


PALT 
115 
249 
2 
125 
192 
133 
96 
874 
We 
Ys, 
Za0 
244 
267 
154 
98 
159 
155 
234 
119 
224 
330 
136 
301 
385 
143 
240 
121 
84 
199 
257 
242 
Zo 
228 
385 
463 
361 
270 


Score 
100 
400 
100 
100 
100 
100 
135 
400 
135 
135 
115 
150 


N6053090C0079 
N6053090C0084 
N6053090C0085 
N6053090C0087 
N6053090C0088 
N6053090C009 I 
N6053090C0092 
N6053090C0095 
N6053090C0096 
N6053090C0097 
N6053090C0098 
N6053090C0099 
N6053090C0100 
N6053090C0102 
N6053090C0105 
N6053090C0108 
N6053090C0109 
N6053090C0113 
N6053090C0114 
N6053090C0115 
N6053090C0116 
N6053090C0117 
N6053090C0118 
N6053090C0119 
N6053090C0122 
N6053090C0125 
N6053090C0127 
N6053090C0128 
N6053090C0129 
N6053090C0133 
N6053090C0134 
N6053090C0135 
N6053090C0148 
N6053090C0149 
N6053090C0153 
N6053090C0155 
N6053090C0158 
N6053090C0161 
N6053090C0162 
N6053090C0163 
N6053090C0167 
N6053090C01 68 
N6053090C0169 


FP 
FP 
EP 
FP 
FP 
CR 
er 
FP 
FP 
FP 
CR 


FP 


FP 
FP 
FP 
FP 
FP 


FP 


CR 
FP 
FP 
jell eg 
FP 
FP 
FP 
FP 
FP 
FP 
FP 


yy 


FP 
FP 


BUU UU 


SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SERV 


SUPP 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


SERV 
SUPP 
SERV 


SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SERV 
SERV 
SUPP 


SUPP 
SERV 
SUPP 
SUPP 
SUPP 


100624 
586404 

42652 
251960 

55700 
921222 
589141 


118624.97 


58674 
747606 


1807978 


27968 
62440 
215988 
34300 
173958 
274929 
32055 
52763 
394656 
644191 
353156 
52037 
93050 
30550 
119412 


65285 .04 


ae) T Taye 
99545 
499248 
151000 
69230 
42900 
251904 
235872 
46000 
49998 
49992 
74412 
47250 
290706 


595515600 


497180 


38 


COMP 
COMP 


COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
COMP 
SS 

COMP 


N6053090C0170 
N6053090C0171 
N6053090C0172 
N6053090C0173 


N6053090C0176. 


N6053090C0178 
N6053090C0179 
N6053090C0180 
N6053090C0184 
N6053090C0185 
N6053090C0186 
N6053090C0188 
N6053090C0189 
N6053090C0190 
N6053090C0191 
N6053090C0192 
N6053090C0194 
N6053090C0195 
N6053090C0201 
N6053090C0203 
N6053090C0204 
N6053090C0207 
N6053090C0208 
N6053090C0209 
N6053090C0210 
N6053090C0212 
N6053090C0215 
N6053090C0217 
N6053090C0218 
N6053090C0219 
N6053090C0222 
N6053090C0223 
N6053090C0224 
N6053090C0225 
N6053090C0227 
N6053090C0228 
N6053090C0229 
N6053090C0232 
N6053090C0233 
N6053090C0235 
N6053090C0236 
N6053090C0237 
N6053090C0238 


CR 


Ox 


CR 


FP 


RD 

RD 

SERV 
SERV 
SUPP 
SUPP 
RD 

RD 

SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
RD 

SERV 
SUPP 
RD 

SUPP 
SUPP 
RD 

SUPP 


68650 
74355 
52000 


7707283 


397579 
359873 
699196 
599694 
130000 
439980 
45500 
35280 
24168 
32594 
47710 
34320 
66140 
123610 
49980 


1044498 


63576 
57660 
497120 
ay to 
Tis67 
215688 


1272 


348433 
35000 


506546.07 


32000 
531264 
497883 

75623 
240942 

30887 

31800 
980371 
248518 

99755 

51000 

44160 

58800 


3, 


N6053090C0239 
N6053090C0240 
N6053090C0244 
N6053090C0245 


N6053090C0246: 


N6053090C0248 
N6053090C0249 
N6053090C0250 
N6053090C0251 
N6053090C0253 
N6053090C0254 
N6053090C0255 
N6053090C0256 
N6053090C0259 
N6053090C0263 
N6053090C0265 
N6053090C0266 
N6053090C0269 
N6053090C0271 
N6053090C0272 
N6053090C0273 
N6053090C0274 
N6053090C0276 
N6053090C0277 
N6053090C0278 
N6053090C0280 
N6053090C0283 
N6053090C0284 
N6053090C0285 
N6053090C0287 
N6053090C0288 
N6053090C0289 
N6053090C0293 
N6053090C0294 
N6053090C0295 
N6053090C0297 
N6053090C0298 
N6053090C0299 
N6053090C0300 
N6053090C0325 
N6053090C0326 
N6053090C0334 
N6053090C0336 


soUGuGU UU 


FP 


SUPP 601785 
SUPP 563952 
SUPP 974586 
SUPP 35420 
SUPP 28000 
RD 55282 
RD 50000 
SUPP 28400 
SUPP 47960.10 
SUPP 517046 
SUPP = 101130 
SUPP 50255 
SUPP 13873313 
SUPP 83882 
SERV 499990 
SUPP = 81250 
SUPP 28390 
SERV 41000 
SUPP = 65500 
SUPP 124488 
SUPP 55023 
SUPP = 133530 
SUPP 52000 
SUPP 38872 
SUPP 61544 
SUPP 26995 
SUPP 47155 
SUPP 49952 
SUPP 81440 
RD 432080 
SUPP 61000 
SUPP 27700 
SUPP 106240 
SUPP 33197 
SUPP 28000 
SUPP 89803.72 
SUPP 377982 
SUPP = 75000 
SUPP 522842 
SUPP 241330.89 
SUPP 99950 
SUPP 156110 
SUPP 484467 


40 


N6053090C0337 
N6053090C0340 
N6053090C0342 
N6053090C0343 
N6053090C0344 
N6053090C0345 
N6053090C0348 
N6053090C0349 
N6053090C035 1 
N6053090C0352 
N6053090C0353 
N6053090C0354 
N6053090C0356 
N6053090C0360 
N6053090C0362 
N6053090C0363 
N6053090C0364 
N6053090C0367 
N6053090C0369 
N6053090C0370 
N6053090C0371 
N6053090C0373 
N6053090C0374 
N6053090C0375 
N6053090C0376 
N6053090C0377 
N6053090C0381 
N6053090C0388 
N6053090C0389 
N6053090C0391 
N6053090C0393 
N6053090C0396 
N6053090C0397 
N6053090C0399 
N6053090C0401 
N6053090C0403 
N6053090C0404 
N6053090C0405 
N6053090C0406 
N6053090C0407 
N6053090C0409 
N6053090C0410 
N6053090C0411 


FP 
Fp 


FP 


CR 


SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SERV 
SERV 
SUPP 
SERV 


SUPP 


SUPP 


SUPP 


SUPP 
SUPP 
SUPP 
SERV 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


SUPP 
SUPP 
SUPP 
SERV 
SUPP 
SUPP 


SUPP 
RD 
RD 
RD 


46255 
39040 
136435 
4538901 
77920 
196650 
543831 
50040 
97606 
799672 
857742 
47500 
131845 
325000 


47236.50 


49999 
30868 
30365 
218000 
80490 
61802 


37800 - 


34800 
38826 
50640 
46000 
95000 
35210 
48240 
41440 
289922 
41500 


129847 
89736 


32834.32 


36920 
50000 
70000 
145230 
44832 
49930 
53194 


41 


N6053090C0412 
N6053090C0413 
N6053090C0414 
N6053090C0415 
N6053090C0416 
N6053090C0417 
N6053090C0418 
N6053090C0419 
N6053090C0424 
N6053090C0425 
N6053090C0427 
N6053090C0428 
N6053090C0430 
N6053090C0435 
N6053090C0436 
N6053090C0445 
N6053090C0446 
N6053090C0447 
N6053090C0448 
N6053090C0449 
N6053090C0450 
N6053090C045 1 
N6053090C0455 
N6053090C0456 
N6053090C0462 
N6053090C0465 
N6053090C0470 
N6053090C0472 
N6053090C0473 
N6053090C0477 
N6053090C0480 
N6053090C0482 
N6053090C0483 
N6053090C0486 
N6053090C0487 
N6053090C0488 
N6053090C0489 
N6053090C0490 
N6053090C0491 
N6053090C0493 
N6053090C0494 
N6053090C0500 
N6053090C0502 


FP 


esMesMesmesme’ 
Dodd tg 


PIUIU UU OUD 


EP 


is 


FP 
FP 
FP 
FP 
FP 
FP 
FP 
FP 
_P 
1 
FP 
FP 
FP 
CR 
FP 


CR 


SERV 


SUPP 


SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


SUPP 
SUPP 
RD 

SERV 


SUPP 
RD 

SUPP 
SUPP 
SUPP 
SUPP 
SUPP 


50000 
518162 
50073 
43921 
381692 
49974 
70002 
26275 
49921 
50000 
37426 
32957 
298345 
36945 
87901 
56884 
50000 
69500 
649996 
49717 
493921 
49512 
143752 
131856 
46700 


24116.50 


34339 
64554 
85914 


858328.92 


49939 
43385 
450200 


1676684 
49826.76 


51150 


48198.75 


712018 
84867 
29475 
69000 

464300 
29110 


42 


N6053090C0507 
N6053090C0508 
N6053090C0509 


FP 
Fp 
CR 


N6053090CO131 FP 


N6053090D0010 
N6053090D0013 
N6053090D0020 
N6053090D0022 
N6053090D0026 
N6053090D0027 
N6053090D0028 
N6053090D0038 
N6053090D0055 
N6053090D0058 
N6053090D0080 
N6053090D0089 
N6053090D0090 
N6053090D0107 
N6053090D0126 
N6053090D0136 
N6053090D0152 
N6053090D0156 
N6053090D0159 
N6053090D0199 
N6053090D0333 


CR 
1S es 
| 
FP 
FP 
EP 
FP 
EP 
FP 


SUPP 
SUPP 
RD 

SUPP 
SUPP 
SUPP 
SUPP 
SERV 
SUPP 
SERV 
SERV 
SUPP 
SERV 
SUPP 
SERV 
RD 

RD 


125000 
39500 
463681 
621210 
25491.55 
W227. 
120678 
119850 
7500000 
2490000 
TAZ 1:28 
152168 
80880 
2826656 
2573400 
1159999 
699706 


SERV 16507057 


SERV 
SERV 
SUPP 
SERV 
SUPP 
SUPP 
SERV 


141336.95 
2908282 
599000 
3311795 
153863.65 
322225 
438400 


43 


Contract Number Amount 


N6053091C0003 
N6053091C0011 
N6053091 C0013 
N6053091C0018 
N6053091C0019 
N605309 1 C0020 
N6053091C0029 
N6053091C0038 
N605309 1C0042 
N6053091C0044 
N6053091C0046 
N6053091C0047 
N6053091C0048 
N6053091C0051 
N6053091C0052 
N6053091C0060 
N6053091C0070 
N6053091C0074 
N605309 1C0076 
N6053091C0077 
N6053091C0087 
N6053091C0096 
N6053091C0097 
N6053091C0098 
N6053091C0104 
N6053091C0105 
N6053091C0107 
N6053091 C0111 
N6053091C0112 
N6053091CO115 
N6053091C0120 
N6053091C0121 
N6053091C0123 
N6053091C0124 
N6053091C0126 
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APPENDIX E: COMPLEXITY POINT STRUCTURE 


P/O NO:N60530-9 AMOUNT: § 


SIMPLIFIED PURCHASE COMPLEXITY POINT STRUCTURE 


wr a ey eS cm ee pe ce ce ee SE fe Se > eee SE SE 
a a am a ee a ee etm a ee ee <a a ee ee eee Oe ES eS 


1. POINTS 
i ALL INCOMING ACTIONS 
2. POINTS ADDITIONAL CONSIDERATIONS 
1 Written RFQ 
1 Sole Source 
2 R & D 
2 NTE 
1 Attachments\Exhibits 
1 Written Amemdments 
2 Unauthorized Commitment 


CANCELLATIONS: Credit will be given for ALL 
points accumulated at the time of cancellation 
4. POINTS AWARD PROCESS 





Each 5 line items (Award CLINS 
Less than 2500 

2501 - 10 K 

10,001 - 25 K 

More than 25 K (GSA) 

NOTE: If an "F" code & commodity code starts with 
"70" then exclude from Simplified Purchase 

- goes to Major Contracts 


NP WH 


5. POINTS MODIFICATIONS 
4 Claim 
2 Bilateral 
1 Unilateral/Admin 


TOTAL POINTS: 


FILE NAME:SMFINAL.WK1 REVISED 4/20/92 


-ONTRACT NO:N60530-9_ - 


AMOUNT: $ 


MAJOR CONTRACTS COMPLEXITY POINT STRUCTURE - PRE AWARD 
RECEIPT/REVIEW OF A R P 




















100 UNDER 500 K 
150 501 K - 1M 
300 1M - 10 M 
400 10,001 M - 50 M 
500 OVER 50 M 
100 SBIR (All phases-Incl. $$) 
SBIR MAX = 100 PTS. + ADMIN. 
a TYPES OF CONTRACTS 
POINTS: 
25 ALL COS T TYPES 
50 R_& D - FIXED PRICE ONLY 
15 ‘ADPE” Requiring Life Cycle 
or Desirable Options 
25 CLASSIFIED MATERIALS 
Ee Oe ee 
POINTS: J&A 
25 UNDER $1 M 
35 $1M+ - $10 M 
50 OVER $10 M 
25 PREPROPOSAL CONFERENCE 
KKKKKKKEKKKKKKKKKKKKKKKKRKKKKKK KKK KKK 
25 DISTRIBUTION TO 


15 OR MORE VENDORS 
(MAILING LIST) 
70% OF POINTS TO HERE FOR EACH 2ND 
OR ADDTL AWARD FOR MULTIPLE AWARDS 
(EX: BAA OR DEVELOPMENT. SPLITS AND 
SBIRS ARE NOT INCLUDED) 


4. POST RFP & FIRST BUS CL. 
(EXCLUDES SBIRS) 
POINTS: 
100 GREATEST VALUE (SOURCE SEL) 
10 AMENDMENTS (EACH) 
25 PROTESTS 
50 TURNIP FACTOR (EG:KR W/O 


APPR. RATES,ACCTG SYSTEM, 
PREV GOVT OR COST TYPE CONTRACT) 





10 AMENDMENTS (EACH) 
15 FACE-TO-FACE 
# IN COMPETITIVE RANGE: 
25 1 KR WITHOUT J & A 
25 MORE THAN 3 OFFERORS 
(NO SBIRS) 
25 PROTEST 
6. POST BUSINESS CLEARANCE 
5 WRITE IT 
5 PRE-AWARD SURVEY 
25 PROTEST 
7 AWARD 
POINTS: 
25 PROTEST 
5 SPLIT AWARD (EACH) 


ADJUSTMENTS (PLUS/MINUS) 
$$ THRESHOLD CHANGE 
TYPE OF CONTRACT CHANGE 


RETURNS / CANCELLATIONS: 
(PERCENTAGE OF POINTS ACCUMULATED) 
AFTER #3 30% CREDIT ONCE 

RFP IS COMPLETED 
AFTER #4 45% CREDIT 
AFTER #5 60% CREDIT 
AFTER #6 90% CREDIT 
DEFINITION: 


PROTEST = TO GAO/GSBCA/DISTRICT 
COURT. MULTIPLE OFFEROR PROTESTS 
AT THE SAME STAGE (ANY STAGE OF 
PROCESS) = 25 POINT TOTAL 


NOTE: POINTS ABOVE THE *** LINE WILL BE AWARDED UPON RECEIPT OF THE ARP 


FOR PROCUREMENT 
FILE NAME: REVCSPTS.WK1 


TOTAL 


REVISED: 4/20/92 


\ conTRACT NO:N60530-9_ - AWD AMT:$ 


MAJOR CONTRACTS COMPLEXITY POINT STRUCTURE - ADMINISTRATION 


BASIC 
ADMINISTRATION 
— TYPES OF CONTRACTS > _ 
—S— a l—™ 
25 FP (OTHER THAN R&D) 
(INCL. T&M) 
35 COST (ALL TYPES) 
50 R & D - FIXED PRICE 
(NOT SBIRS) 
2. CUM CONTRACT DOLLAR VALUE 
— © 
0 UNDER $25K 
\ 25 >$25K To <$10M 
200 10M TO <5OM 
450 OVER $50 M 


"ILE NAME: BASICADM.WK1 170 T Ag le 5 REVISED: 4/13/92 





CONTRACT NO:N60530-9_ - 


-MODIFICATION S 


POOO AMOUNT: $ 


MAJOR CONTRACTS COMPLEXITY POINT STRUCTURE - ADMINISTRATION 


TYPE OF CONTRACT 


200 


300 


400 


ALL COS T TYPES 


R & D - FIXED PRICE ONLY 
(NOT SBIRs) 


ADPE Requiring Life Cycle 
or Desirable Options 


SS EE en SE a ee eS ee ee ee ee SS ee Ss ee ee es ee ee 
SSS ee ee ee ee ee ee ee ee ee ee ee ee ee 


DOLLAR VALUE 


2 et 2 Se ee Ss a se EE a 


UNDER 500 K 
501 K- 1M 
1M- 10M 
10,001 M - 50 M 


OVER 50 M 


-NEW PROCUREMENT 


3. J & A 

POINTS: 
25 UNDER $1 M 
35 $1M+ - $10 M 
50 OVER $10 M 


= at ee a ee ee eG a ee ym ps a mee a es a 
a a Sy ee a ee a ee ee ee ee ee a ee, GS eee Se 


FILE NAME: MODNEWPR.WK1 


REVISED: 4/20/92 


\ «TRACT NO: N60530-9__- POO AMOUNT: $ 


MAJOR CONTRACTS COMPLEXITY POINT STRUCTURE - ADMINISTRATION 
MODIFICATIONS - OTHER 





1. POINTS SIMPLE - MISCELLANEOUS 4. POINTS DISPUTES 
5 ADMINISTRATIVE 
50 DISPUTES 
5 INCREMENTAL FUNDING (K/O FINAL DECISION) 
5 AUTHORIZE PHASES 50 APPEALS TO ASBCA OR 
FEDERAL COURT 
2. POINTS MODERATELY COMPLEX 5. POINTS OVERRUNS 
10 AUTH PHASES W/MINOR CHANGES 
EACH 50 UNDER 500K 
CLARIFICATIONS 
100 OVER 500K 
CHANGES IN CLAUSES 
CHANGE IN DELIVERY SCHEDULE 
g 6. POINTS DEFINITIZATION OR 
. ERMINATION NOTICES SETTLEMENT OF 


VARIOUS CONTRACT ACTIONS: 
GFP/STOP WORK 
... CHANGES 
EXERCISE OPTIONS W/O SF 98 
... UNDEFINITIZED CONTRACT 
CHANGE ORDER (UNDEFINITIZED ACTIONS 
CONTRACT ACTIONS) 





SS SS Se ee ee re SS rae TERMINATIONS 
70 UNDER $500 K 
15 EXTENSIVE CHANGES IN SOW 
EACH 105 501 K - 1M 
EXTENSIVE CHANGES IN CLINS 
140 1M - 10 M 
CHANGE IN DELIVERY SCHEDULE . 
W/CONSIDERATION 210 10,001 M - 50 M 
OPTION WITH SF 98 : 280 OVER 50 M 
1E NAME: MODOTHER.WK1 TOTAL: REVISED: 4/20/92 


Ne CONTRACT NO:N60530-9_ _- 


D/O #: AMOUNT: $ 


MAJOR CONTRACTS COMPLEXITY POINT STRUCTURE - DELIVERY ORDERS 


RECEIPT/REVIEW OF D/O 


i le TYPES OF CONTRACT 
POINTS: 
FIXED PRICE: 
5 PRICED IN CONTRACT 
100 NEGOTIATED FOR IND. D/O 


(NOT PRICED ON CONTRACT) 


COST REIMBURSABLE: 


10 UNILATERAL (UNDER $25K) 
-~ 10 FPRA EXISTS - NO DISCUSSIONS 
50 FPRA EXISTS+ DISCUSSIONS 
100 NO FPRA 
T & M: 
5 JUST ISSUE D/O 
10 ACCEPT PROPOSAL-NO DISCUSSION 
50 NEGOTIATE D/O 
2. C R B ISSUES 
S SUBCONTRACT (ANALYSIS REQ) 
5 WRITE BUS. CL. 


DO _FINAL.WK1 TOTAL POINTS: 








KAaRKKKKRKaKKRKKaRKKKKKRKRKKKKRKK KKK 


VISITS III IIIA AAAS 
D/O MODIFICATIONS 


77 oe oo oo ee SS GS ES OS Se |e ee ee es ee Ss OS ee eee Eee 
SS = ec ee, ee SS Sa SS aa se eee eee ee OO ee eee 











1s DESCRIPTION 
POINTS 
5 ADMINISTRATIVE/UNILATERAL 
25 CHANGE ORDERS/ UCAs 
* DEFINITIZE UNDEFINITIZED 
CONTRACT ACTIONS (UCA) 
2». DELIVERY/P 0 P 





POINTS 

* EXTEND DELIVERY OR 
PERIOD OF PERFORMANCE 

15 DELIVERY/P O P CHANGES 
WITH CONSIDERATION 

10 DELIVERY/P O P CHANGES 


WITHOUT CONSIDERATION 


* SEE "TYPE OF CONTRACT" 
(BLOCK 1) 


REVISED: 4/20/92 


FILE NAME: OPTPOINT REVISED: 4/20/92 
WIT 


COMPARISON OF POINTS 
ASSIGNED TO SMALL AND LARGE PURCHASE ACTIONS 


WE LOWERED OUR EXPECTATIONS FOR THE SIMPLEST 
SMALL PURCHASE BUY FROM 1700 BUYS TO 800 PER YEAR 
AND LEFT THE SIMPLEST MAJOR CONTRACT EXPECTATION 
AT 10 NEW CONTRACTS AWARDS PLUS THEIR BASIC 
ADMINISTRATION PER YEAR. USING THE NUMBERING 
SYSTEM THIS GIVES A "STANDARD" OF 1600 "POINTS" 
PER YEAR USING THE FOLLOWING: 








SIMPLIFIED PURCHASE: MAJOR CONTRACTS: 
BASIC PTS OLD FACTOR NEW BASIC PTS OLD FACTOR NEW 
<2500 10 10 1 S BIR 20 5 100 
2500-10K 25 10 3 <500K 20 5 100 
10K-25K 30 10 4 500K-1M 30 5 «i502 
GSA 15 10 2 1M-10M 60 5 300 
10M-50M 80 5 400 
>50M 100 5 500 
R F Q 10 10 1 
S/S 10 10 1 ALL COST 5 5 25 
R & D 20 10 2 + T&M 
NTE 15 10 2 R&D F/P 10 5 50 
ATT/EXH 10 10. 1 ADPE (REQ. 3 5 15 
AMENDS 10 10 1 LIFE CYCLE) 
UNAUTH. COM20 10 2 CLASSIFIED 5 5 25 
COMMITMENTS J & A <1M 5 5 25 
AWARD 10 10 1 1M-10M 7 5 35 
>10M 10 5 50 
PRE-PROPOSAL 
CONF. 5 5 25 
DIST.>15 VEN 5 5 25 
PROTESTS 5 5 25 
SPLIT AWARD 1 5 5 
(EACH) 
TURNIP 10 5 50 
FACTOR 
AMENDMENTS ARE THE FACE TO 3 5 15 
SAME FOR BOTH FACE 
SIMPLIFIED PURCHASE 1 KR W/O 5 5 25 
AND MAJOR CONTRACTS J&A 
MORE THAN 5 5 25 
3 KRS 
POST B/CL 1 Silas 
PRE-AWD 1 5 5 
SURVEY 


ae ee a ee ee ee ee ee ee ee oe a nc a ge cme cet ccc cg cc ce es ee ee Ge eee Oe eee eee 


Based on a recommendation from Sandy Scharn-Stevens, Blaine 
prepared the following for discussion: 


REVISED POINT STRUCTURES DISCUSSED AT 4/20/92 MEETING... 





SIMPLIFIED PURCHASE: | MAJOR CONTRACTS : 
BASIC POINTS BASIC POINTS OLD NEW 
<2500 1 S BIR 100 100 
(MAX + ADMIN) 
2500-10K 3 <500K 100 100 
10K~-25K & 500K-1M 150 150 
GSA 2 1M-10M 300 200 
10M-50M 400 300 
>50M 500 400 
R F Q uf 
S/S 1 ALL COST 25 25 
R & D 2 + T&M 
NTE Ze R&D F/P 50 50 
ATT/EXH 1 ADPE (REQ. 15 15 
AMENDS 1 LIFE CYCLE) 
GREATEST VALUE 100 
(SOURCE SELECTION) 
UNAUTHORIZED 2 CLASSIFIED 25 25 
COMMITMENTS J & A <1M 25 25 
AWARD 1 1M-10M 5 35 
— >10M 50 50 
CONFERENCE 25 25 
DISTR. > 15 
VENDORS 25 25 
AMENDS 10 1O 
PROTESTS 25 25 
SPLIT AWARDS 0 5 
(EACH) 
TURNIP 50 50 
FACTOR 
MODIFICATIONS : FACE TO 15 15 
FACE 
CLAIMS 4 1 KR W/O 25 25 
BILATERAL 2 J&A 
UNILATERAL 1 MORE THAN 25 25 
3 KRS 
POST B/CL 5 5 
PRE-AWD 5 5 
SURVEY 


CHANGE: ADDED "GREATEST VALUE SOURCE SELECTION" WITH A VALUE OF 
100 POINTS. CHANGED THE POINTS FOR $1 MILLION AND MORE BY 
SUBTRACTING OUT 100 POINTS FROM THOSE WHO ARE AWARDED ON A GO/NO- 
GO BASIS. 
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